PROJECT IMPLEMENTATION, MONITORING AND EVALUATION Objectives: To understand the main differences between processes used to prepare a project implementation plan and budget
What is a monitoring and evaluation? A monitoring system can be defined as an observation system for the project managers to verify whether the project activities are happening according to planning and whether means are used in a correct and efficient manner. The system must supply the project management with a continuous flow of information throughout the course of the project to make it possible to take the right decisions. Monitoring is limited to the relation between the implementation of the activities and the results, in which the results are directly and only determined by the project activities.
The evaluation tries to describe the changes in life and wellbeing of the final users. In the best case one tries to compare the situation before the project and after the project and analyse the positive or/and negative evolutions. Often an evaluation contributes to the decision to stop certain activities or to add others. In the evaluation process, external factors that can interfere with the activities to reach a certain impact on the target group are also taken into account.
Planning of monitoring and evaluation
Monitoring and evaluation (M&E) needs to be prepared. Therefore it is necessary to build it into the design of the project and to allocate resources for it from the start. This section looks at M&E aspects to be considered while designing the project. The evaluation process per se will not be presented entirely in this training programme because it is often conducted by an external person to the project team (an evaluator) and sometimes organised by the donor. However, evaluation has to be planned from the beginning of the design, and a specific budget allocated. Basically, Monitoring is about comparing what was originally planned with what actually happens. It tracks progress at each level of the logical framework: activities, outputs, outcomes and impacts (objectives). has four key concepts:
• comparison
• measurement
• verification
• action
Evaluation is essentially a reality test to assess the significance of the project. In particular, it looks at the efficiency, effectiveness, impact, sustainability and relevance of the project given its stated objectives. Evaluation has two specific purposes: accountability and learning. Accountability refers to the obligation of the project to demonstrate to the donor, stakeholders, beneficiaries and others that it was implemented in compliance with its original design, agreed contracts and plans.
Moreover, evaluation offers opportunities to learn about the achievement of results and the performance of the project team. Lessons learned can be applied to future projects. For development projects, three modes of evaluation are particularly relevant: (1) Self-evaluation carried out by members of the project team; (2) Internal evaluation conducted by people from the organization who have no previous links to the project; and (3) External evaluation, which is managed by external evaluators who have no previous links neither with the project being evaluated nor the organisation implementing the project. External evaluations are usually initiated, led and financed by a donor agency.13 Evaluation is thus a periodic assessment, which usually takes place at the middle or at the end of the project whereas monitoring is a continuous internal process that analyses the project’s achievements concerning the outputs to enable project managers to take corrective action when necessary. Differences between monitoring and evaluation:
4.1 Monitoring plans Monitoring is a core management responsibility. It involves collection, analysis and communication concerning the progress of the project and outputs achieved. It identifies actual and potential successes or failures as early as possible, and facilitates timely adjustments to what is being done. It enables the stakeholders to review progress and to propose action to achieve the objectives. As shown below, there are three types of monitoring. They happen at different levels of the logical framework and serve different functions: 1) implementation monitoring is operational: it monitors the activities and outputs; 2) impact monitoring concentrates on the immediate objectives; 3) reporting concerns the concrete (narrative and financial) reports that have to be prepared and submitted periodically to the main stakeholders, particularly donors.
4.1.1 Implementation monitoring (operational)
What is it? Implementation monitoring answers the question: What have we done? It follows what the project produces (goods, works and services) for its beneficiaries. It is used for implementation management. This type of monitoring is mainly an implementation management tool for the project manager, because it makes it possible to check at any time that implementation is on track. It provides information on whether resources are being used correctly to produce the activities, whether activities are being carried out within the planned time frames and whether outputs are obtained and delivered as necessary. Implementation monitoring can be used for short-term project progress reporting (e.g. quarterly or bi-annually).
How to do it The main tools of this type of monitoring are the implementation plan presented in the previous section and the output indicators. Project management must keep track of how the project is spending the budget, using the inputs and carrying out the activities in order to produce the outputs. The use of the logical framework and implementation tools is recommended. The implementation plans (work breakdown structure, calendar of activities and budget) are just estimates of what will happen in the future. They must be reviewed and modified during implementation on the basis of what really happened.
The implementation plan provides the tools to be used for operational monitoring : • Monthly work plan • Work breakdown structure • Calendar of activities • Budget The main idea of implementation monitoring is to compare the plan and budget with the actual results and to report of that for decision. 4.1.2 Impact monitoring (immediate objective)
What is it? This level of monitoring focuses on immediate objectives and their contribution to the development objective during the project’s implementation. Impact monitoring is an input into the final evaluation which will verify the impact of the project only at the end. The key question in this type of monitoring is What have we achieved? The centre of attention is what changes the project has produced in the ultimate beneficiaries and the target group. The development of changes in stakeholders, the ownership of achievements and sustainability are especially relevant aspects of this type of monitoring.
This type of monitoring is a performance management tool, since it provides information on whether outputs are used and owned by the target groups, on the changes that the project is producing on the context, and on any other aspects related to the project’s objectives. This is essential information for the project manager and stakeholders to see whether the project is contributing to development beyond its activities and products.
How to do it Monitoring that focuses on results requires painting the initial picture of the situation that the project will change. This initial situation is called the baseline. It will be useful as a comparison point with which to verify progress towards the results. The baseline paints the initial picture, which is essential in the monitoring of results. Progress is measured using the indicators for the immediate objectives. These are complemented by performance questions on key matters such as ownership, use and usefulness of products, sustainability, compliance by strategic partners and contextual factors.
In this type of monitoring, the participation of stakeholders is crucial, given that they are responsible for the outputs. The timing of data collection is therefore usually more spread out than in implementation monitoring. Reports also cover longer periods, usually annual or biennial.
Tools for impact monitoring Tools to be used for impact monitoring: indicators and means of verification developed in the logical framework and baselines and mid-term targets for achieving those indicators. Usually the information is used for this type of monitoring is presented in an M&E matrix (see below).
4.1.3 Reporting All the information gathered in the monitoring plans will allow you to prepare the interim and final reports. The progress of the project against what was planned is assessed and the information is presented clearly in a report. Specific templates exist for each donor, consistent with the initial application form, so that you can compare what was planned with what was actually achieved. Indicators are therefore crucial to monitoring the success of the project and to reporting on it. 4.2 Planning the evaluation What is it? The evaluation is intended to make an overall assessment of the completed project. The purpose is to determine the relevance of the achievement of the objectives, the effectiveness, the efficiency, the impact and the sustainability of the project.
The evaluator is usually external. He or she will assess the impact of the project, given the planned objectives. The participation of the stakeholders in the evaluation is crucial in order to ensure that the different perspectives and views are taken into account.
In most cases, the evaluation is only conducted at the end of the project, but for wider projects and programmes mid-term and ex-ante evaluations can be conducted. Mid-term evaluations are often similar to impact monitoring but are conducted by an external assessor, whereas monitoring is internal. It gives the project management and the stakeholders an independent analysis of the progress made towards the planned objectives, one with which they can review the strategy. Ex-post evaluation (or impact evaluation) is conducted after the end of the project, up to five years later, in order to verify if the results obtained by the project are sustainable.
How to do it Evaluation, based on the indicators, focusing on the project’s immediate objective and how your project contributes to the development objective.
The logical framework clearly specifies what is to be achieved (outputs and immediate objective), how it is to be verified (indicators and means of verification) and the key assumptions. The project management will prepare the terms of reference (ToR) of the evaluation, based on expected outputs as mentioned in the logframe.
Adequate resources should be set aside for conducting the evaluation (hiring a consultant, field visits, etc.) and the necessary conditions and capacities ensured.
Evaluation is the last step in the project cycle but it is not the end of a project. Indeed, it can be considered the starting point for a new planning process, because the conclusions of the evaluation will allow the stakeholders to draw lessons that may guide future decision-making and project identification.
That is the logic of the project cycle.
When you have finished reading this section and you feel ready, please click on the following icon to start the quiz for that section
Objectives: To understand the main differences between processes used to prepare a project implementation plan and budget
What is a monitoring and evaluation?
A monitoring system can be defined as an observation system for the project managers to verify whether the project activities are happening according to planning and whether means are used in a correct and efficient manner. The system must supply the project management with a continuous flow of information throughout the course of the project to make it possible to take the right decisions. Monitoring is limited to the relation between the implementation of the activities and the results, in which the results are directly and only determined by the project activities.
The evaluation tries to describe the changes in life and wellbeing of the final users. In the best case one tries to compare the situation before the project and after the project and analyse the positive or/and negative evolutions. Often an evaluation contributes to the decision to stop certain activities or to add others. In the evaluation process, external factors that can interfere with the activities to reach a certain impact on the target group are also taken into account.
Planning of monitoring and evaluation
Monitoring and evaluation (M&E) needs to be prepared. Therefore it is necessary to build it into the design of the project and to allocate resources for it from the start. This section looks at M&E aspects to be considered while designing the project.
The evaluation process per se will not be presented entirely in this training programme because it is often conducted by an external person to the project team (an evaluator) and sometimes organised by the donor. However, evaluation has to be planned from the beginning of the design, and a specific budget allocated.
Basically, Monitoring is about comparing what was originally planned with what actually happens. It tracks progress at each level of the logical framework: activities, outputs, outcomes and impacts (objectives). has four key concepts:
• comparison
• measurement
• verification
• action
Evaluation is essentially a reality test to assess the significance of the project. In particular, it looks at the efficiency, effectiveness, impact, sustainability and relevance of the project given its stated objectives. Evaluation has two specific purposes: accountability and learning. Accountability refers to the obligation of the project to demonstrate to the donor, stakeholders, beneficiaries and others that it was implemented in compliance with its original design, agreed contracts and plans.
Moreover, evaluation offers opportunities to learn about the achievement of results and the performance of the project team. Lessons learned can be applied to future projects. For development projects, three modes of evaluation are particularly relevant: (1) Self-evaluation carried out by members of the project team; (2) Internal evaluation conducted by people from the organization who have no previous links to the project; and (3) External evaluation, which is managed by external evaluators who have no previous links neither with the project being evaluated nor the organisation implementing the project. External evaluations are usually initiated, led and financed by a donor agency.13 Evaluation is thus a periodic assessment, which usually takes place at the middle or at the end of the project whereas monitoring is a continuous internal process that analyses the project’s achievements concerning the outputs to enable project managers to take corrective action when necessary.
Differences between monitoring and evaluation:
4.1 Monitoring plans
Monitoring is a core management responsibility. It involves collection, analysis and communication concerning the progress of the project and outputs achieved. It identifies actual and potential successes or failures as early as possible, and facilitates timely adjustments to what is being done. It enables the stakeholders to review progress and to propose action to achieve the objectives.
As shown below, there are three types of monitoring. They happen at different levels of the logical framework and serve different functions:
1) implementation monitoring is operational: it monitors the activities and outputs;
2) impact monitoring concentrates on the immediate objectives;
3) reporting concerns the concrete (narrative and financial) reports that have to be prepared and submitted periodically to the main stakeholders, particularly donors.
4.1.1 Implementation monitoring (operational)
What is it?
Implementation monitoring answers the question: What have we done? It follows what the project produces (goods, works and services) for its beneficiaries. It is used for implementation management. This type of monitoring is mainly an implementation management tool for the project manager, because it makes it possible to check at any time that implementation is on track. It provides information on whether resources are being used correctly to produce the activities, whether activities are being carried out within the planned time frames and whether outputs are obtained and delivered as necessary. Implementation monitoring can be used for short-term project progress reporting (e.g. quarterly or bi-annually).
How to do it
The main tools of this type of monitoring are the implementation plan presented in the previous section and the output indicators. Project management must keep track of how the project is spending the budget, using the inputs and carrying out the activities in order to produce the outputs. The use of the logical framework and implementation tools is recommended. The implementation plans (work breakdown structure, calendar of activities and budget) are just estimates of what will happen in the future. They must be reviewed and modified during implementation on the basis of what really happened.
The implementation plan provides the tools to be used for operational monitoring :
• Monthly work plan
• Work breakdown structure
• Calendar of activities
• Budget
The main idea of implementation monitoring is to compare the plan and budget with the actual results and to report of that for decision.
4.1.2 Impact monitoring (immediate objective)
What is it?
This level of monitoring focuses on immediate objectives and their contribution to the development objective during the project’s implementation. Impact monitoring is an input into the final evaluation which will verify the impact of the project only at the end.
The key question in this type of monitoring is What have we achieved? The centre of attention is what changes the project has produced in the ultimate beneficiaries and the target group. The development of changes in stakeholders, the ownership of achievements and sustainability are especially relevant aspects of this type of monitoring.
This type of monitoring is a performance management tool, since it provides information on whether outputs are used and owned by the target groups, on the changes that the project is producing on the context, and on any other aspects related to the project’s objectives. This is essential information for the project manager and stakeholders to see whether the project is contributing to development beyond its activities and products.
How to do it
Monitoring that focuses on results requires painting the initial picture of the situation that the project will change. This initial situation is called the baseline. It will be useful as a comparison point with which to verify progress towards the results. The baseline paints the initial picture, which is essential in the monitoring of results. Progress is measured using the indicators for the immediate objectives. These are complemented by performance questions on key matters such as ownership, use and usefulness of products, sustainability, compliance by strategic partners and contextual factors.
In this type of monitoring, the participation of stakeholders is crucial, given that they are responsible for the outputs. The timing of data collection is therefore usually more spread out than in implementation monitoring. Reports also cover longer periods, usually annual or biennial.
Tools for impact monitoring
Tools to be used for impact monitoring: indicators and means of verification developed in the logical framework and baselines and mid-term targets for achieving those indicators. Usually the information is used for this type of monitoring is presented in an M&E matrix (see below).
4.1.3 Reporting
All the information gathered in the monitoring plans will allow you to prepare the interim and final reports. The progress of the project against what was planned is assessed and the information is presented clearly in a report. Specific templates exist for each donor, consistent with the initial application form, so that you can compare what was planned with what was actually achieved. Indicators are therefore crucial to monitoring the success of the project and to reporting on it.
4.2 Planning the evaluation
What is it?
The evaluation is intended to make an overall assessment of the completed project. The purpose is to determine the relevance of the achievement of the objectives, the effectiveness, the efficiency, the impact and the sustainability of the project.
The evaluator is usually external. He or she will assess the impact of the project, given the planned objectives. The participation of the stakeholders in the evaluation is crucial in order to ensure that the different perspectives and views are taken into account.
In most cases, the evaluation is only conducted at the end of the project, but for wider projects and programmes mid-term and ex-ante evaluations can be conducted. Mid-term evaluations are often similar to impact monitoring but are conducted by an external assessor, whereas monitoring is internal. It gives the project management and the stakeholders an independent analysis of the progress made towards the planned objectives, one with which they can review the strategy. Ex-post evaluation (or impact evaluation) is conducted after the end of the project, up to five years later, in order to verify if the results obtained by the project are sustainable.
How to do it
Evaluation, based on the indicators, focusing on the project’s immediate objective and how your project contributes to the development objective.
The logical framework clearly specifies what is to be achieved (outputs and immediate objective), how it is to be verified (indicators and means of verification) and the key assumptions. The project management will prepare the terms of reference (ToR) of the evaluation, based on expected outputs as mentioned in the logframe.
Adequate resources should be set aside for conducting the evaluation (hiring a consultant, field visits, etc.) and the necessary conditions and capacities ensured.
Evaluation is the last step in the project cycle but it is not the end of a project. Indeed, it can be considered the starting point for a new planning process, because the conclusions of the evaluation will allow the stakeholders to draw lessons that may guide future decision-making and project identification.
That is the logic of the project cycle.